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Use of edTPA for preservice teacher assessment is becoming increasingly common across the 
country, with some states, including Georgia, mandating the passing of the edTPA for initial 
teacher licensure. This state-wide study investigated whether edTPA as a new policy initiative 
was being integrated by the teacher education programs and faculty in their practice. The 
edTPA Levels of Integration (Lol) Survey was sent to faculty, eliciting 145 responses. Results 
showed that the integration of edTPA was significantly higher at the program level than in 
faculty ’.v personal practice and that, most aspects of edTPA were currently being used at the 
mechanical level. The five-factor solution from the exploratory factor analysis and a high 
Cronbach’s alpha showed that this survey is internally consistent and a valid measure for 
measuring edTPA faculty’s Lol. These factors were robust and strong constructs for 
understanding faculty’s use of edTPA both at the program level and the individual faculty level. 
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The trends in P-12 education related to accountability and testing have had a profound 
impact on teacher education (Darling-Hammond, 2010). A national portfolio assessment 
(edTPA) is now being pushed as a valid and reliable assessment of preservice teachers’ readiness 
to teach (SCALE, 2013) and is already being used across 30 states for different purposes. Policy 
changes in teacher education program requirements involving the use of this tool as a high-stakes 
assessment have been met with both enthusiasm and concerns from faculty, preservice teachers, 
and other stakeholders. Advocates argue for the need for a perfonnance assessment to set the bar 
for expectations for beginning teachers, while critics find standardization of teacher preparation 
and the outsourcing of evaluation to a corporate entity problematic (Sato, 2014). As states have 
moved toward the adoption of new policies requiring edTPA, institutions of higher education 
have been faced with the need to provide intensive faculty professional development and to 
consider extensive curricular redesign to ensure candidates are prepared with the professional 
knowledge and expertise to succeed. In Georgia, policy changes resulted in all institutions of 
higher education being required to implement edTPA in their programs during the 2014-2015 
school year, with the assessment results becoming consequential for certification beginning in 
fall 2015. 

In spring 2015, an edTPA Levels of Integration (Lol) survey was developed and 
distributed to 453 edTPA faculty in 35 institutions across Georgia. The edTPA Lol was 
designed to explain the behaviors of people involved in implementing an innovation (Hall & 
Hord, 2015)—in this case, implementing curricular changes related to elements of the edTPA 
assessment. Although there was an expectation that the institutions would pilot edTPA within 
their programs and make design and delivery changes in preparation for fall 2015, the level of 
integration of edTPA within programs and faculty members’ personal practice was likely to be 
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inconsistent during the implementation year. We were interested in finding out what aspects of 
edTPA were being integrated in the year prior to the consequential year, to what extent, and what 
factors contributed to the different levels of integration. 

In order for researchers to detennine whether a new policy initiative is successful, it is 
important to know how the change is being implemented by the key players in their practice, at 
the grass-roots level (Hall, 2010; Overbaugh & Lu, 2008). In this study, teacher education 
faculty who were teaching methods and/or practicum courses or supervising student teaching in 
their programs were the key implementers of the edTPA policy. The edTPA Lol instrument was 
adapted from two constructs, (a) levels of use, and (b) innovation configuration, drawn from the 
concerns-based adoption model (Hall, 2010). 

The Levels of Use (LOU) construct complicates the use!non-use dichotomy, and provides 
varying levels and degrees of behaviors exhibited by the users of an innovation (Hall & Hord, 
2015). There are three possible kinds of levels of use in the non-users category and five different 
kinds of users. The construct of innovation mapping focuses specifically on the nature of content 
or activities undertaken during implementation of an innovation. Items for the edTPA Lol 
related to the extent to which faculty had been involved in the integration of edTPA and included 
a listing of potential components of what such integration might entail. We also examined 
questionnaires sent to edTPA coordinators by the state agency responsible for the policy change 
of edTPA becoming an initial licensure requirement. In preparation for the implementation year, 
the state agency’s edTPA policy committee and area coordinators surveyed institutions to 
understand the nature of their involvement in edTPA activities and what might be done to 
support their progress. Components addressed in these questionnaires were used as the basis for 
creation of items for the present survey. 
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We created a 6-level edTPA Lol survey (see Appendix A) consisting of two aspects: Lol 
at the program level and Lol at the individual faculty level, within courses. We envision the 
edTPA Lol model as following a developmental trajectory; as faculty learn more about edTPA, 
the extent of the information they have and the degree to which they integrate change in their 
personal practice detennines their movement along these developmental stages. Given the 
national trends in adopting the edTPA as a high-stakes assessment (Sato, 2014), it is critical to 
understand how edTPA is being integrated within the programs and also within faculty 
members’ personal practices. 

We hypothesize that Lol at the program and individual faculty level could be different. It 
is possible for a program to recommend its use, but personal philosophies, concerns, or other 
circumstances can obstruct integration of edTPA components in faculty practice. Conversely, an 
individual faculty member may have the motivation to try out edTPA, but the overall acceptance 
for edTPA may be lesser at the program level. Awareness about edTPA integration at the 
program and faculty’s personal levels would provide information about Georgia’s status of 
edTPA preparation prior to the consequential year, in addition to detennining the professional 
development needs of the faculty. 

Because the edTPA Lol survey did not have an established reliability and validity, this 
study addressed the following research questions: (a) Is the edTPA Lol a valid and reliable 
measure of faculty’s integration of edTPA?, and (b) How do edTPA faculty describe their 
integration of edTPA, both within their programs and their personal practice? 

Thus, the purpose of this study was to determine how teacher education faculty respond 
to policy implementation, and how policy is integrated within the program and personal practices 
of the faculty who were responsible for teaching the courses most impacted by edTPA. The Lol 
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survey can potentially serve as a tool to understand integration of policy components at the 
program and course level, and can help facilitation of professional development to address 
specific needs of faculty. 

Literature Review 

Implementation of educational reform is complex. In order to evaluate whether a reform 
is successful, a more nuanced understanding of the degree to which the refonn is being used at 
the grass-roots level is needed (Hall, 2010; Overbaugh & Lu, 2008). edTPA Lol is grounded in 
previous research about the concerns-based adoption model of an innovation, which leads us to 
suggest that faculty’s integration of edTPA will be related to the degree to which faculty use 
aspects of edTPA in their courses, make changes in curriculum, and integrate it within their 
practices (Hall, 2010; Hall, Dirksen, & George, 2008). The edTPA Lol complicates the use!non¬ 
use dichotomy and provides a more complete understanding of the nature of edTPA integration 
within program and faculty’s personal practice. Such nuanced information on impact of edTPA 
roll-out on the practice of teacher educators is valuable for policymakers as it allows for deeper 
understanding of how edTPA integration will unfold in a state if it is used for high-stakes. For 
instance, within the non-use realm, it is helpful to understand if a faculty member is integrating 
edTPA at the orientation stage or the preparation state. Similarly, the distinction among users is 
important to understand because the faculty at the mechanical level would be using edTPA very 
differently as compared to faculty at the refinement or integration stages. The mechanical stage 
is characterized by insufficient mastery over the innovation and disjointed use, with focus on 
complying with the demands, and making adaptations in personal work and time to fit the 
innovation in their schedule. In contrast, the refinement and integration stages are characterized 
by an understanding of the big-picture and the context of change. At this stage, faculty’s focus 
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shifts away from themselves as implemented of the change to seeking benefit for their students 
and making adaptations in their practice informed by assessment data. Based on this 
information, targeted professional development and support can be provided to faculty to 
facilitate their movement into the user realm, ideally helping them reach the refinement stage 
(Hall & Hord, 2015). 

Teacher education has long relied on locally developed assessments that lack reliability 
and validity (Castle, & Shaklee, 2006; Grossman, Hammemess, McDonald, & Ronfeldt, 2008). 
Rigorous performance-based assessments for preservice teachers have been advanced as one 
possible way to ensure that all students receive instruction from a high-quality teacher (Darling- 
Hammond, 2010). edTPA is a performance-based assessment focusing on the application of 
knowledge of teaching and learning in a classroom setting (Wei & Pecheone, 2010). Currently, 
640 educator preparation programs in 35 states are participating in edTPA use (SCALE, 2015). 
Across states, variations are evident with respect to edTPA use. Some states use this assessment 
for certification, others for program approval, while some others for both. Additionally, different 
states have detennined different cut-off scores to signify a passing performance on edTPA; there 
has also been variation in the timeline for implementation of edTPA as a policy in these states, 
prior to edTPA scores becoming consequential. 

The implementation of edTPA has been especially controversial because of critiques 
from the field about the developmental appropriateness of the edTPA tasks for preservice 
teachers, minimizing collaborative nature of learning in interaction with the cooperating teacher 
and the field supervisor, and its replacement with high-stakes, labor intensive portfolio (Lit & 
Lotan, 2013; Margolis & Doring, 2013). Furthennore, the directions about the appropriate levels 
of support that the faculty or supervisors can afford to the teacher candidates during the edTPA 
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portfolio development phase has been inconsistently understood and utilized by the field, adding 
to the confusion about appropriate uses of instruction around edTPA (Gitomer & Bell, 2013; 
Okhremtchouk, Newell, & Rosa, 2013; Sandholtz & Shea, 2012). However, in a state where the 
policy has been issued for edTPA to be used for licensure, the faculty have limited options but to 
prepare the candidates for this assessment. The concerns about edTPA voiced by others in the 
field, in addition to their personal concerns about the edTPA can obstruct faculty’s integration of 
edTPA in their practices. 

Because the edTPA is a relatively new assessment and had morphed from its previous 
version: PACT and TPA (SCALE, 2013), research is still coming out about the process and 
consequences of the use of edTPA for high-stakes. The trend of edTPA’s state-wide 
consequential use is also recent, and there is a lack of research in the field to guide edTPA 
implementation across universities and programs. There is also a need for survey instruments 
and protocols that focus specifically on edTPA as a policy, and the challenges it poses for the 
teacher education community in identifying areas needing support and professional development 
to facilitate smoother transition to policy. 

Methods 

Next, we describe the edTPA Lol survey, which has 6 levels of integration and describes 
integration of edTPA within the program and at the level of individual faculty members’ 
personal practice. 

Measures 

The edTPA Lol online survey included items constructed in light of the Levels of Use 
(LoU) instrument (Hall, Dirksen, & George, 2008), as well as research about how configurations 
of an innovation can be described (Hall, 2010). Utilizing items from a state questionnaire sent to 
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edTPA coordinators describing institutional activities and needs, we developed 19 questions and 
situated them within the level of use and innovation configuration constructs (Hall & Hord, 
2015). Questions 1-10 focused on Lol at program level, while questions 11-19 focused on 
faculty’s personal Lol (Appendix A). The questions for the program Lol were parallel to the 
personal Lol (except question 1). Responses from these 19 items were expected to provide 
insight into faculty’s behavior in relation to edTPA use (Hall, et ah, 2008). These items were 
rated on a six-point Likert Scale (Nonuse: 1, Orientation: 2, Preparation: 3, Mechanical Use: 4, 
Refinement: 5, and Integration: 6). The progression on the 6-level scale was developmental and 
reflected the stages of integration individuals go through when faced with a new innovation. 
Levels 1-3 were within the realm of non-use but included different levels of 
awareness/preparation, while Level 4-6 represented varying levels of use (Hall, 2010). 

In addition, we also asked the faculty responding to the survey to provide some 
background information, which included: name of the edTPA handbook they used in their 
courses, the size of institution (based on the number of initial teacher preparation graduates 
reported in Title II), and the roles that these faculty played in their institutions (student teaching 
supervisor, methods course instructor, or both). Our hypothesis was that edTPA integration 
might look different across the different edTPA content handbooks, across small and large 
institutions in the state, and might also be different when faculty played different roles in their 
institutions. 

Participants 

A total of 453 teacher education faculty in 35 institutions across the state of Georgia were 
e-mailed the online edTPA Lol survey. A total of 145 faculty responded to the survey in spring 
2015. As shown in Table 1, the responses represented faculty perspectives on integrating a 
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variety of the edTPA content handbooks: elementary, middle grades, secondary (English, Math, 
Science and Social Studies), special education, and P-12 (art, music, foreign language, and 
physical education). Some of the faculty were involved in integration of more than one edTPA 
handbooks. 

Table 1. 

edTPA Handbooks Used by Faculty 


Programs 


N 

Elementary 


61 

Middle Grades 

Math 

30 


Social Studies 

34 


English 

32 


Science 

31 

Secondary 

English 

34 


Math 

25 


Science 

28 


Social Studies 

36 


Special Education 

15 

P-12 (art, music, foreign language, and physical education) 

38 


Analyses 

We conducted exploratory factor analysis (EFA) to examine how well the items clustered 
together and whether survey items measured what they were expected to measure. Prior to 
conducing EFA, Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy and Bartlett’s test 
of Sphericity assessed the suitability of the data for factor analysis. 

In order to find an optimal number of factors, we assessed cumulative percentage of 
variance and eigenvalue (>1.0), scree plot and model fit indices (e.g., RMSEA, CFI, TLI) using 
two software programs: SPSS (v. 20) and Mplus (Ver.7) (Muthen & Muthen, 1998). In this 
analysis, we applied an oblique promax rotation. In order to determine the model fit and the 
factor retention, the criteria of RMSEA values between 0.05 and 0.08, CFI (>0.90), and TLI 
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(>0.90), were considered as acceptable fit indices (Fabrigar, Wegener, MacCallum, & Strahan, 
1999; Garver & Mentzer, 1999). A reliability analysis was also conducted to confirm the 
consistency of items within each factor. Then we conducted the pair-wise comparison of Levels 
of Integration, in order to examine the difference how edTPA faculty describe their integration of 
edTPA within their program and their personal practice. 

Results 

Descriptive Analysis 

Responses to the 19 questions were examined for nonnal distribution using descriptive 
statistics. Missing values were dealt with list-wise deletion as a default option. Based on the 
criteria of |2.0| for skewness and |7.0| kurtosis (Chou & Bentler, 1995; Curran, West, & Finch, 
1996), none of items showed violations of normality (Table 2) and an approximately normal 
distribution was evident for the data. 
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